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I.  INTRODUCTION: 


Phase  II  of  the  Integrated  “Clinical  Information  System  Collaboration  Project:  IHI  Trigger 
Tool”  has  been  completed.  The  contract  term  ended  February  28,  2011.  The  proposal 
for  this  study  included  work  in  progress  from  Phase  I.  Data  collection  for  phase  I  has 
been  completed  along  with  a  comparison  and  analysis  of  data  pre  and  post-CPOE. 

Objective:  Identify  adverse  events  as  they  relate  to  the  identification  of  harm  or  injury  to 
the  patient  due  to  medication  administration. 

Hypothesis:  CPOE  with  decision  support  decreases  the  probability  of  adverse  events 
caused  by  medication  administration. 

This  project  is  Phase  II  in  the  “Integrated  Clinical  Information  System  Collaboration 
Project.”  Phase  I  of  the  project  included  a  surrogate  measurement  for  adverse 
medication  outcomes  through  the  collection  of  data  on  the  number  of  major 
contraindicated  medication  alerts.  A  high  level  alert  indicates  possible  serious  drug-drug 
interactions  that  will  likely  result  in  patient  harm.  Pre-CPOE  only  the  pharmacist  saw 
these  alerts.  Post-CPOE  the  physicians  saw  these  alerts  initially  and  were  expected  to 
act  on  them.  If  the  physicians  did  not  take  action,  the  alerts  would  then  be  seen  by  the 
pharmacist.  In  the  Post-CPOE  environment,  this  dual  monitoring  system,  by  physician 
and  pharmacist,  is  seen  as  a  potential  way  of  reducing  potential  harmful  ADE’s. 

Although  Mission  Hospital  assessed  mortality  rates  along  with  ADEs  in  Phase  I,  there 
was  no  provision  for  effectively  measuring  ADEs  in  relation  to  mortality  rates.  For  phase 
II  of  this  study  Mission  Hospital  used  a  tool  developed  by  The  Institute  for  Health  Care 
Improvement  (IHI).  This  is  a  Trigger  Tool  for  identifying  ADEs  with  harm.  This  tool 
includes  a  list  of  known  ADE  triggers  and  instructions  for  collecting  the  data  needed  to 
assess  the  number  of  ADEs  per  1 ,000  doses  and  the  percentage  of  admissions  with 
ADEs.  Mission  Hospital  obtained  permission  from  the  Institute  for  Health  Care 
Improvement  (IHI)  to  use  this  tool  to  assess  and  compare  patient  harm  due  to 
medication  errors  pre  and  post  CPOE. 

II.  BODY: 

The  Institute  for  Healthcare  Improvement’s  (IHI)  trigger  tool  process  consisted  of  a 
retrospective  patient  chart  review  completed  by  a  team  of  nursing  staff  receiving  training 
and  direction  from  a  staff  physician.  Charts  were  reviewed  using  a  structured  approach 
to  identify  significant  medication  events  with  harm.  Based  on  the  large  size  of  the 
comparison  samples,  a  convenience  sampling  methodology  was  used  to  insure 
randomness.  Samples  were  pulled  based  on  discharge  dates  from  a  representative 
time  period  both  before  and  after  CPOE  implementation.  In  order  to  eliminate  any 
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seasonal  bias,  the  time  periods  were  consistent  for  both  samples.  With  approximately 
3,300  admissions  per  month,  a  timed  based  convenience  sample  should  minimize  any 
other  bias  as  the  “population”  for  that  time  period  will  become  the  sample. 


Statistical  Analysis: 

A  statistical  sample  was  calculated  using  Minitab  14  statistical  software.  A  two 
proportion  sample  size  calculation  was  used  to  determine  the  appropriate  sample  size 
for  comparison  before  and  after  CPOE  implementation.  An  assumed  pre-CPOE  defect 
rate  of  8%  was  used  for  the  hypothesized  defect  rate,  with  a  post-CPOE  rate  of  6%, 
indicating  the  ability  of  the  sample  size  to  detect  a  change  of  2%.  The  samples  were 
calculated  using  an  alpha  (significance)  level  of  .05  and  a  statistical  power  of  0.9.  The 
resulting  sample  size  based  on  the  preceding  criteria  would  be  3,419  for  EACH 
population.  For  this  study  3,450  patient  charts  were  reviewed  for  each  of  two  sample 
periods. 

Study  Design: 

This  tool  counted  only  ADEs  with  harm  to  the  patient  from  medications  whether  or  not 
they  resulted  in  an  identifiable  error.  Harm  was  defined  as  “temporary  or  permanent 
impairment  of  physical  or  psychological  body  function  or  structure.”  Based  upon  the  IHI 
error  classification  scheme,  the  tool  excluded  the  following  categories  and  described 
medication  errors  that  do  not  cause  harm: 


Category  A:  Circumstances  or  events  having  the  capability  to  cause  error 
Category  B:  An  error  not  reaching  the  patient 
Category  C:  An  error  reaching  the  patient  but  not  causing  harm 
Category  D:  An  error  reaching  the  patient  and  requiring  monitoring  or  intervention  to 
confirm  resulting  in  no  harm  to  the  patient 


The  tool  included  categories  E,  F,  G,  H,  and  I  and  described  medication  errors  that  do 
cause  harm. 


Category  E: 
Category  F: 

Category  G: 
Category  H: 
Category  I: 


Temporary  harm  to  the  patient  and  requiring  intervention 
Temporary  harm  to  the  patient  and  requiring  initial  or  prolonged 
hospitalization 
Permanent  patient  harm 
Intervention  requiring  sustaining  life 
Patient  death 
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The  IHI  provides  a  list  of  triggers  to  be  useful  in  identifying  ADEs.  A  review  of  patient 
charts  was  conducted  and  a  team  of  registered  nurses  along  with  the  PI  worked 
together  to  agree  on  the  IHI  list  of  triggers  and  added  a  few  other  triggers  they  deemed 
appropriate  for  the  study  (appendix  A). 

Once  the  team  decided  on  this  list  of  triggers  the  next  step  was  to  review  a  sample  of 
patient  records.  Each  patient  record  review  resulted  in  the  generation  of  a  form-based 
report,  whether  or  not  the  record  turned  out  to  contain  triggers  and  ADEs  (appendix  B). 
The  team  reviewed  randomly  selected  patient  charts  from  those  patients  admitted  for 
greater  than  two  days.  Historical  data  was  collected  and  analyzed  to  compare  pre-  and 
post-CPOE  implementation.  At  the  completion  of  25  reviewed  charts,  a  summary  sheet 
was  completed  (appendix  C). 

Results: 

The  following  measures  were  calculated  based  upon  aggregated  ADE  assessment 
data: 

•  Total  ADEs  per  1 ,000  Doses  (table  1 ) 

•  The  total  number  of  ADEs  per  1 ,000  doses  was  calculated  as  the  total  number 
of  ADE’s  identified  in  a  sample  of  inpatient  records,  divided  by  the  total  number 
of  medication  doses  administered  to  those  patients.  The  result  is  then  multiplied 
by  1,000. 


Pre-  Vs.  Post-CPOE  ADE's  per  1,000  doses 

Total 

ADE's 

Total  Medication 

Doses 

%  per  1,000  Doses 

Pre-CPOE 

302 

646,948 

.46  events/1,000 
doses 

Post-CPOE 

257 

669,449 

.38  events  /  1,000 
doses 

(Table  1) 

P  value  =  0.021 


•  The  percent  of  Inpatient  admissions  associated  with  any  ADE  (table  2) 

•  The  total  number  of  ADE’s  was  calculated  as  the  number  of  inpatient 

admissions  associated  with  an  ADE,  divided  by  the  total  number  of  admission 
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encounters  for  the  selected  time  period,  multiplied  by  100  and  expressed  as  a 
percentage. 


Percent  of  Inpatient  Admissions  with  an  ADE 
3,450  Consecutive  patients  per  year 


Pre-CPOE  /  2007  Post-CPOE  /  2008 


Total 

%  of 

Total 

%  of 

2007-2008 

Admissions 

Admissions 

Admissions 

Admissions 

Change 

273 

7.91% 

235 

6.81% 

-1.1% 

(Table2) 

P  value  =  0.080 


•  Total  ADEs  per  inpatient  admission  (table  3) 

•  The  total  number  of  ADEs  per  inpatient  admission  was  calculated  as  the  number 
of  total  ADE’s,  all  harm  categories,  divided  by  the  total  number  of  admission 
encounters  for  a  selected  time  period,  multiplied  by  100  and  expressed  as  a 
percentage. 


Percent  of  Patient  Admissions  with  an  ADE 
3,450  Consecutive  patients  per  year 

Pre-CPOE  /  2007 

Post-CPOE  /  2008 

Total 

%  of 

Total 

%  of 

2007-2008 

Admissions 

Admissions 

Admissions 

Admissions 

Change 

302 

8.75% 

257 

7.45% 

-1.3% 

(Table  3) 


P  value  =  0.047 

•  Total  Category  E  ADEs  per  inpatient  admission  (table  4) 

•  The  total  number  of  Category  E  ADEs  was  calculated  as  the  number  of 
inpatient  admissions  associated  with  a  category  E  ADE,  temporary  harm 
requiring  intervention,  divided  by  the  total  number  of  admission  encounters  for 
a  selected  time  period,  multiplied  by  100  and  expressed  as  a  percentage. 


Pre-CPOE  /  Total  Consecutive  Patients  each  Period  =  3,450 


Category 

No.  of  ADE's 

%  of  Total  Patients 

2007-2008  Change 

Pre-CPOE 

E 

254 

7.36% 

Post-CPOE 

E 

211 

6.12% 

-1.23% 

(Table  4) 
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P  value  =  0.039 


Inpatient  admission  associated  with  a  category  E  ADE  per  encounter  (table  5) 

•  The  total  number  of  individual  patients  who  were  identified  as  having  category 
E  harm  per  inpatient  admission  was  calculated  as  the  number  of  category  E 
ADEs,  temporary  harms  requiring  intervention,  divided  by  the  total  number  of 
admission  encounters  for  a  selected  time  period,  multiplied  by  100  and 
expressed  as  a  percentage. 


Pre-CPOE  /  Total  Consecutive  Patients  each  Period  = 

3,450 

Category 

No.  of  ADE's 

%  of  Total  Patients 

2007-2008  Change 

Pre-CPOE 

E 

226 

6.55% 

Post-CPOE 

E 

189 

5.48% 

-1.07% 

(Table  5) 


P  value  =  0.061 

Total  Category  G  ADEs  per  inpatient  admission  (table  6) 

•  The  total  number  of  Category  G  ADEs  was  calculated  as  the  number  of 
inpatient  admissions  associated  with  a  category  G  ADE,  those  causing 
permanent  patient  harm,  divided  by  the  total  number  of  admission  encounters 
for  a  selected  time  period,  multiplied  by  100  and  expressed  as  a  percentage. 


Pre-CPOE  /  Total  Consecutive  Patients  each  Period 

=  3,450 

Category 

No.  of  ADE's 

%  of  Total  Patients 

2007-2008  Change 

Pre-CPOE 

G 

1 

0.03% 

Post-CPOE 

G 

2 

0.06% 

0.03% 

(Table  6) 


P  value  =  0.564 

•  Total  Category  H  ADEs  per  inpatient  admission  (table  7) 

The  total  number  of  Category  H  ADEs  was  calculated  as  the  number  of 
inpatient  admissions  associated  with  a  category  G  ADE,  those  requiring 
intervention  to  sustain  life,  divided  by  the  total  number  of  admission 
encounters  for  a  selected  time  period,  multiplied  by  100  and  expressed  as 
a  percentage. 


Pre-CPOE  /  Total  Consecutive  Patients  each  Period  =  3,450 


Category 

No.  of  ADE's 

%  of  Total  ADEs 

2007-2008  Change 

Pre-CPOE 

G 

7 

0.20% 

Post-CPOE 

G 

5 

0.14% 

-0.06% 

(Table  7) 


P  Value  =  0.563 

In  summary,  total  ADE’s  with  harm  were  assessed  Pre-  vs.  Post-CPOE.  Incidents  by 
severity  are  shown  in  tables  one  and  two.  Overall  there  were  302  total  ADEs  that 
occurred  in  273  patients’  pre-CPOE  and  a  total  of  257  total  ADEs  that  occurred  in  235 
patients’  post-CPOE  (tables  8  and  9). 


Total  Number  of  ADEs 

Total  Consecutive  Patients  =  3,450 

Pre-CPOE 

Post-CPOE 

Category 

Total  No.  of 

ADE's 

%  of  Total  ADEs/ 
Patient  Admission 

Total  No.  of 

ADE’s 

%  of  Total  ADEs/ 
Patient  Admission 

2007-2008 

Change 

E 

254 

7.36% 

211 

6.12% 

(1.2%) 

F 

37 

1.07% 

36 

1.04% 

(0.0%) 

G 

1 

0.03% 

2 

0.06% 

0.0% 

H 

7 

0.20% 

5 

0.14% 

(0.1%) 

1 

3 

0.09% 

3 

0.09% 

0.0% 

Total 

302 

8.75% 

257 

7.45% 

(1.3%) 

(Table8) 


Total  Number  of  Inpatient  Admissions  Associated  with  an  ADE 

Total  Consecutive  Patients  =  3,450 

Pre-CPOE 

Post-CPOE 

Category 

Total  No.  of 

ADE's 

%  of  Total  ADEs/ 
Patient  Admission 

Total  No.  of 

ADE’s 

%  of  Total  ADEs/ 
Patient  Admission 

2007-2008 

Change 

E 

226 

6.55% 

189 

5.48% 

(1.1%) 

F 

36 

1.04% 

36 

1.04% 

0.0% 

G 

1 

0.03% 

2 

0.06% 

0.0% 

H 

7 

0.20% 

5 

0.14% 

(0.1%) 

1 

3 

0.09% 

3 

0.09% 

0.0% 

Total 

273 

7.91% 

235 

6.81% 

(1.1%) 

(Table  9) 
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Individual  reviewers  classified  events  by  patient  preventable  harm  to  determine  severity 
by  incident.  The  number  of  ADEs  assessed  per  1 ,000  doses  or  number  of  ADEs  per 
admission  yielded  a  statistically  significant  difference  indicating  fewer  ADEs  post  versus 
pre  CPOE.  Only  when  the  percent  of  admissions  was  analyzed  for  the  occurrence  of 
any  ADE  did  our  results  fall  short  of  statistical  significance. 

The  distribution  of  severity  of  ADEs  weighted  heavily  toward  harms  considered  to  be 
temporary.  Categories  E  and  F  made  up  8.43%  of  the  patient  population  pre-CPOE  and 
7.1%  post-CPOE. 

The  occurrence  of  temporary  harm  in  subcategory  E  showed  a  lower  incidence  post 
CPOE  that  reached  statistical  significance.  Category  F  harm  remained  relatively 
unchanged  pre-  versus  post-CPOE.  G-l  Harm,  considered  to  be  permanent  or  life- 
threatening,  were  not  found  to  reflect  statistically  significant  differences  between  the  two 
groups.  The  incidence  of  most  serious  harm,  category  F-l,  was  far  less  common,  such 
that  no  discernable  difference  could  be  assessed  pre-  vs.  post  CPOE.  There  were  an 
equal  number  of  category  I  deaths  related  to  medication  administration  during  the 
course  of  the  study,  three  pre-CPOE  and  three  post-CPOE. 

Although  the  differences  in  ADEs  pre-  versus  post-  CPOE  are  relatively  small,  the  large 
number  of  records  reviewed  in  this  study  indicates  that  CPOE,  even  early  in  the  course 
following  its  deployment,  results  in  a  lower  incidence  of  ADEs.  These  changes  were 
evident  within  the  first  year  of  CPOE  when  physician  experience  using  the  system  was 
still  recent  and  when  there  had  been  little  deployment  of  supporting  rules  or  decision 
support.  These  results  showing  early  positive  results  suggest  that  the  addition  of 
increasingly  sophisticated  decision  support  may  be  expected  to  yield  greater  benefit 
over  time.  In  this  context,  CPOE  should  be  viewed  as  a  critical  piece  of  the  EHR 
foundation  that  may  be  expected  to  result  in  a  continuing  improvement  in  medication 
safety. 

III.  KEY  RESEARCH  ACCOMPLISHMENTS: 

•  Assessment  of  the  impact  of  CPOE  on  ADEs  in  a  large  patient  population 

•  Analysis  of  ADEs  by  severity  subcategory  and  by  both  admission  occurrences 
and  per  1,000  doses. 

IV.  REPORTABLE  OUTCOMES: 

There  are  no  manuscripts,  abstracts,  presentations;  patents  and  licenses  applied  for 
and/or  issued;  degrees  obtained  that  are  supported  by  this  award;  development  of  cell 
lines,  tissue  or  serum  repositories;  informatics  such  as  databases  and  animal  models, 
etc.;  funding  applied  for  based  on  work  supported  by  this  award;  employment  or 
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research  opportunities  applied  for  and/or  received  based  on  experience/training 
supported  by  this  award  at  this  time. 

V.  CONCLUSION: 

In  this  study,  we  evaluated  the  rates  and  type  of  ADEs  pre-  vs.  post-CPOE  and  found 
that  the  overall  rates  were  fairly  similar.  It  was  determined  that  Mission  Hospital  had 
comparable  incident  rates  of  ADE’s  as  compared  to  previous  studies1. 

Evaluation  of  medication  harms  suggests  that  the  majority  of  harm  can  be  eliminated  by 
use  of  decision  support  tools  and  protocols.  This  study  did  not  show  significant 
reductions  in  drug  harm  with  the  use  of  CPOE.  Further  studies  should  be  done  to 
include  harms  to  determine  impact  on  length  of  stay  and  cost  per  case.  In  addition, 
these  harms  were  not  assessed  by  provider  type.  In  some  cases  the  preventable  harm 
could  be  attributed  to  direct  patient  care  of  nursing  staff.  These  studies  could  increase 
awareness  of  nursing  issues  concerning  patient  medication  administration  and 
monitoring  post  medication  delivery. 

VI.  APPENDICES: 


AppendixA.  docx  AppendixB.docx  AppendixC.  docx 


VII.  QUARTERLY  AND  YEAR  TO  DATE  EXPENDITURES: 

Reporting  period  from  10/01/2010 _ to  02/28/201 1 _ 
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Current  staff,  with  percent  effort  of  each  on  project: 
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Karen  Roby,  Project  Manager  80 _ % 
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